Medical image processing apparatus, medical image processing method and medical image processing system

ABSTRACT

A medical image processing apparatus includes an acquisition unit and a processing unit. The acquisition unit acquires volume data of a subject. The processing unit displays a three-dimensional image by rendering the acquired volume data, on a display unit. The processing unit displays a first object showing (i) a point on a body surface of the subject and (ii) a direction to the volume data in the three-dimensional image, and displays a two-dimensional image of a surface including the point on the body surface and being defined based on the direction, in the volume data. The processing unit acquires information of a first operation to change display of the two-dimensional image, and moves the point on the body surface along the body surface of the subject based on the first operation to update display of the first object and the two-dimensional image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority based on Japanese Patent ApplicationNo. 2018-138000, filed on Jul. 23, 2018, the entire contents of whichare incorporated by reference herein.

TECHNICAL FIELD

The present disclosure relates to a medical image processing apparatus,a medical image processing method and a medical image processing system.

BACKGROUND ART

In recent years, there is a technique to generate an image based onvolume data obtained by a computed tomography (CT) device withinformation obtained by an ultrasound probe in a real space. Forexample, in US 2014/0187919 discloses to acquire positional coordinatesof an ultrasound probe in a real space and to generate a multi planerreconstruction (MPR) image corresponding to the acquired positionalcoordinates (see US 2014/0187919).

In recent years, a technique for virtually generating an image obtainedusing an ultrasound probe has become common.

SUMMARY OF INVENTION

In some cases, diagnosis and operation plans are made using CT data, andan operation is performed in accordance with pre-operative planningusing an ultrasound image in an actual operation. The techniquedisclosed in US 2014/0187919 is applied to generation of a virtualultrasound image, so that it is possible to acquire positionalcoordinates of an ultrasound probe in a virtual space and generate anMPR image corresponding to the acquired positional coordinates duringthe operation. Here, pre-operative planning may include a place to betouched by the ultrasound probe before an operation. When a doctor makesdiagnosis, it is easier to understand accurate position of disease orthe like in a subject when the subject is observed using atwo-dimensional image than when the subject is observed using athree-dimensional image. A three-dimensional image is useful when theentire subject is observed from above. Therefore, for example, in a casewhere a user operates a two-dimensional image to update display whileremembering an ultrasound image, it is preferable to ascertain to whichposition and which direction the changed two-dimensional imagecorresponds in a three-dimensional image.

The present disclosure is contrived in view of the above-describedcircumstances and provides a medical image processing apparatus, amedical image processing method and a medical image processing systemwhich are capable of ascertaining to which position and which directiona changed two-dimensional image corresponds in a three-dimensional imagein a case where a user operates the two-dimensional image to updatedisplay thereof.

According to one aspect of the disclosure, a medical image processingapparatus includes an acquisition unit and a processing unit. Theacquisition unit acquires volume data of a subject. The processing unitdisplays a three-dimensional image by rendering the acquired volumedata, on a display unit. The processing unit displays a first objectshowing (i) a point on a body surface of the subject and (ii) adirection with respect to the volume data in the three-dimensionalimage, on the display unit. The processing unit displays atwo-dimensional image of a surface on the display unit. The surfaceincludes the point on the body surface and is defined based on thedirection, in the volume data. The processing unit acquires informationof a first operation to change display of the two-dimensional image. Theprocessing unit moves the point on the body surface along the bodysurface of the subject based on the first operation to update display ofthe first object and the two-dimensional image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a hardware configuration example of amedical image processing apparatus in a first embodiment;

FIG. 2 is a block diagram showing a functional configuration example ofthe medical image processing apparatus;

FIG. 3A is a diagram showing an example of movement of a virtual probeon a body surface of a subject shown by a three-dimensional image;

FIG. 3B is a diagram showing an example of rotation of a virtual probeon a body surface of a subject shown by a three-dimensional image;

FIG. 4A is a diagram showing an example of a 3D virtual probe;

FIG. 4B is a diagram showing an example of a 2D virtual probe;

FIG. 5 is a diagram showing an example of a positional relationshipbetween a subject, a virtual probe, and an MPR surface in athree-dimensional space;

FIG. 6 is a diagram showing a display example using a display;

FIG. 7 is a diagram showing an example in which a scale display issuperimposed on an MPR image;

FIG. 8 is a flowchart showing an example of an outline of operations ofthe medical image processing apparatus;

FIG. 9 is a diagram for supplementing description of operations of themedical image processing apparatus according to each operation on an MPRimage;

FIG. 10 is a flowchart showing an operation example of the medical imageprocessing apparatus during a slice paging operation on an MPR image;

FIG. 11 is a diagram illustrating slice paging according to a slicepaging operation in a comparative example;

FIG. 12 is a diagram illustrating slice paging according to a slicepaging operation in the first embodiment;

FIG. 13 is a diagram showing an example of transition of an MPR imageand a 2D virtual probe according to a slice paging operation on an MPRimage in the first embodiment;

FIG. 14 is a diagram showing an example of transition of athree-dimensional image and a 3D virtual probe according to a slicepaging operation on an MPR image in the first embodiment;

FIG. 15 is a flowchart showing an operation example of the medical imageprocessing apparatus during a panning operation on an MPR image;

FIG. 16 is a diagram showing transition of an MPR image and a 2D virtualprobe according to a panning operation in a comparative example;

FIG. 17 is a diagram showing an example of transition of an MPR imageand a 2D virtual probe according to a panning operation on the MPR imagein the first embodiment;

FIG. 18 is a diagram showing an example of transition of athree-dimensional image and a 3D virtual probe according to a panningoperation on an MPR image in the first embodiment;

FIG. 19 is a flowchart showing an operation example of the medical imageprocessing apparatus during a rotation operation on an MPR image;

FIG. 20 is a diagram showing transition of an MPR image and a 2D virtualprobe according to a rotation operation in a comparative example;

FIG. 21 is a diagram showing an example of transition of an MPR imageand a 2D virtual probe according to a rotation operation on the MPRimage in the first embodiment; and

FIG. 22 is a diagram showing an example of transition of athree-dimensional image and a 3D virtual probe according to a rotationoperation on an MPR image in the first embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of the present disclosure will be describedwith reference to the accompanying drawings.

In the present disclosure, a medical image processing apparatus includesan acquisition unit, a processing unit and a display unit. Theacquisition unit acquires volume data of a subject. The processing unit:displays a three-dimensional image by rendering the acquired volumedata, on the display unit; displays a first object showing (i) a pointon a body surface of the subject and (ii) a direction with respect tothe acquired volume data in the three-dimensional image, on the displayunit; displays a two-dimensional image of a surface on the display unit,the surface including a point on the body surface and being definedbased on the direction, in the volume data; acquires information of afirst operation to change display of the two-dimensional image; andmoves the point on the body surface along the body surface of thesubject based on the first operation to update display of the firstobject and the two-dimensional image.

According to the present disclosure, a user observes the subject usingthe two-dimensional image and easily understands an accurate position ofdisease or the like in the subject. The user can observe the entiresubject from above by using the three-dimensional image. In this case,the user can ascertain to which position and which direction thetwo-dimensional image corresponds in the three-dimensional image.

First Embodiment

FIG. 1 is a block diagram showing a configuration example of a medicalimage processing apparatus 100 in a first embodiment. The medical imageprocessing apparatus 100 includes a port 110, a user interface (UI) 120,a display 130, a processor 140, and a memory 150.

A computed tomography (CT) scanner 200 is connected to the medical imageprocessing apparatus 100. The medical image processing apparatus 100acquires volume data from the CT scanner 200 and processes the acquiredvolume data. The medical image processing apparatus 100 may include apersonal computer (PC) and software mounted on the PC.

The CT scanner 200 irradiates an internal organism with X-rays to obtainan image (CT image) using a difference in the absorption of X-rays dueto tissues in the body. Examples of the internal organism include ahuman body and the like. The internal organism is an example of asubject.

A plurality of CT images may be obtained in time series. The CT scanner200 generates volume data including information on any portion insidethe internal organism. Any portion inside the internal organism mayinclude various internal organs (for example, a brain, a heart, akidney, a large intestine, a small intestine, a lung, a breast, mammaryglands, and prostate glands). By acquiring the CT image, voxel values(CT values) of voxels in the CT image are obtained. The CT scanner 200transmits volume data as the CT image to the medical image processingapparatus 100 via a wired circuit or a wireless circuit.

The CT scanner 200 includes a gantry (not shown) and a console (notshown). The gantry includes an X-ray generator (not shown) and an X-raydetector (not shown), and performs imaging at a predetermined timingindicated by the console to detect X-rays having passed through a humanbody and obtain X-ray detection data. The X-ray generator includes anX-ray tube (not shown). The console is connected to the medical imageprocessing apparatus 100. The console acquires a plurality of pieces ofX-ray detection data from the gantry and generates volume data based onthe X-ray detection data. The console transmits the generated volumedata to the medical image processing apparatus 100. The console mayinclude an operation unit (not shown) for inputting patient information,imaging conditions related to CT imaging, contrast conditions related toadministration of a contrast medium, and other information. Theoperation unit may include an input device such as a keyboard or amouse.

The CT scanner 200 can also acquire a plurality of pieces ofthree-dimensional volume data by continuously performing image togenerate a moving image. Data regarding a moving image based on aplurality of pieces of three-dimensional volume data is also referred toas four-dimensional (4D) data.

The CT scanner 200 may obtain CT images at a plurality of timings. TheCT scanner 200 may obtain a CT image in a state where a subject isimaged. The CT scanner 200 may obtain a CT image in a state where asubject is not imaged.

The port 110 within the medical image processing apparatus 100 includesa communication port and an external device connection port and acquiresvolume data obtained from a CT image. The acquired volume data may beimmediately transmitted to the processor 140 to be subjected to variousprocessing, or may be stored in the memory 150 and then transmitted tothe processor 140 to be subjected to various processing. In addition,the volume data may be acquired through a recording medium or recordingmedia.

The volume data imaged by the CT scanner 200 may be transmitted from theCT scanner 200 to an image data server (picture archiving andcommunication systems: PACS) (not shown) and stored therein. The port110 may acquire the volume data from the image data server instead ofacquiring the volume data from the CT scanner 200. In this manner, theport 110 functions as an acquisition unit that acquires various datasuch as volume data.

The UI 120 may include a touch panel, a pointing device, a keyboard, ora microphone. The UI 120 receives any input operation from a user of themedical image processing apparatus 100. The user may include a doctor, aradiology technician, or other paramedic staffs.

The UI 120 receives operations such as designation of a region ofinterest (ROI) and setting of luminance conditions in the volume data.The region of interest may include regions of various tissues (forexample, blood vessels, a bronchus, an internal organ, a bone, a brain,a heart, a foot, a neck, and a blood flow). The tissues may broadlyinclude tissues of an internal organism such as lesion tissues, normaltissues, internal organs, and organs. In addition, the UI 120 mayreceive operations such as designation of a region of interest andsetting of luminance conditions in volume data or an image based on thevolume data (for example, a three-dimensional image and atwo-dimensional image to be described later).

The display 130 may include a liquid crystal display (LCD) and displaysvarious pieces of information. The various pieces of information mayinclude a three-dimensional image and a two-dimensional image obtainedfrom volume data. The three-dimensional image may include a volumerendering image, a surface rendering image, a virtual endoscope image(VE image), a virtual ultrasound image, a curved planar reconstruction(CPR) image, and the like. The volume rendering image may include aray-sum image (also referred to simply as a “SUM image”), a maximumintensity projection (MIP) image, a minimum intensity projection (MinIP)image, an average value (average) image, or a ray-cast image. Thetwo-dimensional image may include an axial image, a sagittal image, acoronal image, a multi planer reconstruction (MPR) image, and the like.The three-dimensional image and the two-dimensional image may include acolor fusion image.

The memory 150 includes various primary storage devices such as a readonly memory (ROM) and a random access memory (RAM). The memory 150 mayinclude a secondary storage device such as a hard disk drive (HDD) and asolid state drive (SSD). The memory 150 may include a tertiary storagedevice such as a USB memory or an SD card. The memory 150 stores variouspieces of information and programs. The various pieces of informationmay include volume data acquired by the port 110, an image generated bythe processor 140, setting information set by the processor 140, andvarious programs. The memory 150 is an example of a non-transitoryrecording medium in which programs are recorded.

The processor 140 may include a central processing unit (CPU), a digitalsignal processor (DSP), or a graphics processing unit (GPU). Theprocessor 140 functions as a processing unit 160 performing variousprocesses and control by executing a medical image processing programstored in the memory 150.

FIG. 2 is a block diagram showing a functional configuration example ofthe processing unit 160.

The processing unit 160 includes a region extraction unit 161, an imagegeneration unit 162, a virtual probe processing unit 163, an operationinformation acquisition unit 164, and a display control unit 166.

The processing unit 160 controls units of the medical image processingapparatus 100. The units included in the processing unit 160 may work asdifferent functions by one piece of hardware or may be realized asdifferent functions by a plurality of pieces of hardware. In addition,the units included in the processing unit 160 may work by dedicatedhardware parts.

The region extraction unit 161 may perform segmentation processing involume data. In this case, the UI 120 receives an instruction from auser, and the instructed information is transmitted to the regionextraction unit 161. The region extraction unit 161 may performsegmentation processing from the volume data by a general method basedon the instructed information and may extract (segment) a region ofinterest. In addition, a region of interest may be set manually by auser's detailed instruction. Further, in a case where an object to beobserved is determined in advance, the region extraction unit 161 mayperform segmentation processing from the volume data without a user'sinstruction or may extract a region of interest including an object tobe observed. The region to be extracted may include regions of varioustissues (for example, blood vessels, a bronchus, an internal organ, abone, a brain, a heart, a foot, a neck, a blood flow, mammary glands, abreast, and a tumor).

The region extraction unit 161 may extract a body trunk of a subject psas a region of interest. The region extraction unit 161 may extract thebody trunk of the subject ps in accordance with, for example, regiongrowing. The body trunk may correspond to, for example, a trunk portionof the subject ps or may be a portion including a breast and an abdomen.The body trunk may include areas related to the body trunk of thesubject ps such as a head portion, a trunk portion, an arm portion, anda foot portion.

The image generation unit 162 may generate a three-dimensional image ora two-dimensional image based on volume data acquired by the port 110.The image generation unit 162 may generate a three-dimensional image(for example, a raycast image) or a two-dimensional image (for example,an MPR image) from the volume data acquired by the port 110 based on adesignated region or a region extracted by the region extraction unit161.

The virtual probe processing unit 163 generates a virtual probe pr as aUI object which shows an ultrasound probe used in a real space. Thevirtual probe pr may include a 3D virtual probe pr1 displayed togetherwith a three-dimensional image. The 3D virtual probe pr1 may bedisplayed so as to be superimposed on the three-dimensional image. The3D virtual probe pr1 comes into contact with the subject ps representedby a three-dimensional image in the three-dimensional image in a virtualspace and is movable on a body surface of the subject ps. The 3D virtualprobe pr1 may be moved along the body surface of the subject ps.

The virtual probe pr may include a 2D virtual probe pr2 displayedtogether with a two-dimensional image. The 2D virtual probe pr2 may bedisplayed so as to be superimposed on the two-dimensional image. The 3Dvirtual probe pr1 displayed together with the three-dimensional imagecorresponds to the 2D virtual probe pr2 displayed together with thetwo-dimensional image. That is, the position and direction of the 3Dvirtual probe pr1 on the three-dimensional space match and correspond tothe position and direction of the 2D virtual probe pr2 in atwo-dimensional plane.

Therefore, in a case where the position and direction of the 3D virtualprobe pr1 in the three-dimensional space change, the position anddirection of the 2D virtual probe pr2 in a two-dimensional plane maychange. In contrast, in a case where the position and direction of the2D virtual probe pr2 in the two-dimensional plane change, the positionand direction of the 3D virtual probe pr1 in the three-dimensional spacemay change. The virtual probe processing unit 163 determines theposition and direction of the 3D virtual probe pr1 so that the 3Dvirtual probe pr1 displayed together with the three-dimensional imagemaintains a contact with the body surface of the subject ps representedby the three-dimensional image even when a display range of thetwo-dimensional image in the two-dimensional plane changes.

The operation information acquisition unit 164 acquires information ofvarious operations on a two-dimensional image through the UI 120.

The various operations may include an operation (also referred to as aslice paging operation) to move a two-dimensional image showing anycross-section (also referred to as, for example, an MPR surface SF or aslice) in volume data. The slice paging operation may be an operation tochange the MPR surface SF to another MPR surface SF parallel to the MPRsurface SF. The operation information acquisition unit 164 may detectthe acquisition of slice paging for an MPR image G1 by a slider GUI (notshown) as an example of the UI 120 detecting, for example, a slide in avertical direction.

The various operations may include an operation to move atwo-dimensional image in any cross-section (within a cross-section) involume data (also referred to as a panning operation). That is, thepanning operation may be an operation to move a display range of the MPRsurface SF in parallel in the MPR image G1 of the MPR surface SF. Theoperation information acquisition unit 164 may acquire a panningoperation on the MPR image G1 by the UI 120 detecting a draggingoperation on the MPR image G1 displayed on the display 130.

The various operations may include an operation to rotate any image inany cross-section in volume data (also referred to as a rotationoperation). That is, the rotation operation may be an operation torotate the MPR image G1 without changing the MPR surface SF of the MPRimage G1. The operation information acquisition unit 164 may acquire arotation operation by the UI 120 detecting a dragging operation forrotation with respect to a rotation instructing unit rp (see FIG. 4B) ofthe 2D virtual probe pr2 displayed on the display 130.

The image generation unit 162 may generate a three-dimensional image ora two-dimensional image based on operation information acquired by theoperation information acquisition unit 164. The virtual probe processingunit 163 may generate the virtual probe pr (the 3D virtual probe pr1 andthe 2D virtual probe pr2) based on operation information acquired by theoperation information acquisition unit 164.

The display control unit 166 displays various data, information, andimages on the display 130. The display control unit 166 may display thethree-dimensional image or the two-dimensional image generated by theimage generation unit 162. The display control unit 166 may display thevirtual probe pr generated by the virtual probe processing unit 163. Inthis case, the display control unit 166 may display the 3D virtual probepr1 so as to be superimposed on the three-dimensional image. The displaycontrol unit 166 may display the 2D virtual probe pr2 so as to besuperimposed on the two-dimensional image.

In this manner, the display control unit 166 may visualize as athree-dimensional image or a two-dimensional image based on volume dataacquired from the CT device 200. The display control unit 166 mayvisualized show the 3D virtual probe pr1 and the 2D virtual probe pr2 soas to indicate a positional relationship between the three-dimensionalimage and the two-dimensional image.

In the present embodiment, a raycast image is mainly exemplified as athree-dimensional image G2, but another three-dimensional image may beused. The MPR image G1 is mainly exemplified as a two-dimensional image,but another two-dimensional image may be used.

The three-dimensional image G2 may include a three-dimensional image G2Aon which the 3D virtual probe pr1 moving during a slice paging operationis superimposed, a three-dimensional image G2B on which the 3D virtualprobe pr1 moving during a panning operation is superimposed, and athree-dimensional image G2C on which the 3D virtual probe pr1 rotatingduring a rotation operation is superimposed. The MPR image G1 mayinclude an MPR image G1A obtained during a slice paging operation, anMPR image G1B obtained during a panning operation, and an MPR image G1Cobtained during a rotation operation.

Next, an example of movement of the 3D virtual probe pr1 on thethree-dimensional image G2 will be described.

FIG. 3A is a diagram showing an example of movement of the 3D virtualprobe pr1 on a body surface psf of the subject ps shown by thethree-dimensional image G2. FIG. 3B is a diagram showing an example ofmovement of the 3D virtual probe pr1 on the body surface psf of thesubject ps shown by the three-dimensional image G2. In FIG. 3B, the bodysurface psf of the subject ps is not shown.

The 3D virtual probe pr1 may move on the body surface psf in accordancewith an operation on the MPR image G1 through the UI 120. The 3D virtualprobe pr1 may move on the body surface psf in accordance with anoperation on the three-dimensional image G2 through the UI 120.

The virtual probe pf1 may move along the body surface psf of the subjectps. The virtual probe pf1 may be rotated on the body surface psf of thesubject ps. Therefore, the virtual probe pf1 is not separated from thebody surface psf, and at least one point of the virtual probe may bebrought into contact with the body surface psf or the virtual probe maybe brought into contact with the body surface psf in a surface contactmanner. The virtual probe pf1 can be moved or rotated while maintaininga contact with the body surface psf.

As shown in FIG. 3A, the 3D virtual probe pr1 is movable on the bodysurface psf with two degrees of freedom in u and v directions. As shownin FIG. 3B, the 3D virtual probe pr1 is rotatable on the body surfacepsf with three degrees of freedom in a θ direction, a φ direction, and aΨ direction.

A rotation direction of the 3D virtual probe pr1 received in thethree-dimensional image G2 is may limit to any two directions or onedirection (for example, the Ψ direction) among the above-described threedirections. Since the degree of freedom of the 3D virtual probe pr1 inthe rotation direction is limited, it becomes easy for a user operatingthe 3D virtual probe pr1 to intuitively know rotation through theoperation, whereby operability is improved.

FIGS. 4A and 4B are diagrams showing an example of the virtual probe pr(the 3D virtual probe pr1 and the 2D virtual probe pr2). The 3D virtualprobe pr1 shown in FIG. 4A is displayed together with athree-dimensional image. The 2D virtual probe pr2 shown in FIG. 4B isdisplayed together with the MPR image G1.

Although the 3D virtual probe pr1 and the 2D virtual probe pr2 may bethe same three-dimensional UI object, the virtual probes are obliquelyviewed in FIG. 4A and viewed in a side view in FIG. 4B, and thus thevirtual probes are viewed differently between FIGS. 4A and 4B. Thevirtual probe pr has an upper surface us and a lower surface ds, and itis assumed that ultrasound waves are virtually transmitted along adirection from the upper surface us to the lower surface ds. That is, itis assumed that the virtual probe pr emits virtual ultrasound waves.Since the lower surface ds of the virtual probe pr comes into contactwith the body surface psf of the subject ps, virtual ultrasound wavesare transmitted from the lower surface ds to the inside of the body ofthe subject ps (the inner side of the body surface psf illustrated in acylindrical shape). A passage through which the virtual ultrasound wavespass may be represented by a one-dimensional straight line or a regionin a two-dimensional plane (parallel to a straight line) including thestraight line. This two-dimensional plane is an MPR surface SF.

FIG. 5 is a diagram showing an example of a positional relationshipbetween the subject ps, the virtual probe pr, and the MPR surface SF ina three-dimensional space.

In FIG. 5 , the subject ps is illustrated as a cylinder and the virtualprobe pr is illustrated as a plate. In FIG. 5 , it is assumed thatvirtual ultrasound waves transmitted from the virtual probe pr moveinside the body of the subject ps along an extension line of a plateillustrating the 3D virtual probe pr1. In this case, this extension lineis a path of the virtual ultrasound waves and along with the MPR surfaceSF. The appearance on the MPR surface SF is shown as an MPR image. Thatis, the MPR image is equivalent to a virtual ultrasound image.

FIG. 6 is a diagram showing a display example using the display 130.

The display 130 displays a set including the three-dimensional image G2and the 3D virtual probe pr1 and a set including the MPR image G1 andthe 2D virtual probe pr2. The display 130 may simultaneously display thethree-dimensional image G2, the 3D virtual probe pr1, the MPR image G1,and the 2D virtual probe pr2. Thereby, the user can easily ascertaindetails of the MPR image G1 and the three-dimensional image G2 andascertain which position and direction of the subject ps in thethree-dimensional image G2 correspond to the MPR image G1. Accordingly,for example, the user can recognize which organ of the subject ps isseen in the MPR image G1 generated based on the virtual probe pr, bybringing the virtual probe pr into contact with a certain position ofthe subject ps in a certain direction.

The display 130 may display the set including the three-dimensionalimage G2 and the 3D virtual probe pr1 and the set including the MPRimage G1 and the 2D virtual probe pr2 at different timings. The display130 may not display the 2D virtual probe pr2 corresponding to the MPRimage G1.

FIG. 7 is a diagram showing an example in which a scale display issuperimposed on the MPR image G1.

In FIG. 7 , a distance from the body surface psf of the subject pscoming into contact with the 2D virtual probe pr2 is displayed in anoverlapping manner. Here, as an example, a range from 0 mm to 100 mm isshown as information of the distance. In this manner, the medical imageprocessing apparatus 100 superimposes a scale display on the MPR imageG1, so that the user can easily confirm, for example, a distance to anaffected area of an observation target in the subject ps. The user caneasily confirm whether or not there is an importance organ or bloodvessel on a straight line on which virtual ultrasound waves transmittedfrom the 2D virtual probe pr2 move. The medical image processingapparatus 100 can run a simulation of puncture while confirming a scaledisplay, and it is possible to improve safety in puncture. Treatment ofthe puncture may include burning of the affected area. A scale may bedisplayed obliquely from the side of the 2D virtual probe pr2, assuminga case of an endoscopic ultrasound aspiration needle. In this manner,the medical image processing apparatus 100 may make pre-operativeplanning by performing a scale display on the MPR image G1.

Next, operations of the medical image processing apparatus 100 accordingto an operation on the MPR image G1 will be described.

FIG. 8 is a flowchart showing an example of an outline of operations ofthe medical image processing apparatus 100.

First, the port 110 acquires volume data of the subject ps from the CTdevice 200 and the like (S11). The subject ps is, for example, a humanbody.

The region extraction unit 161 generates contour of a body trunk basedon the volume data including the human body (S12). In this case, theregion extraction unit 161 may extract volume data in a range surroundedby the contour of the body trunk (that is, volume data of the bodytrunk) from the volume data including the human body.

The image generation unit 162 visualizes the 3D body trunk of thesubject ps with a raycast method (S13). That is, the image generationunit 162 may generate the three-dimensional image G2 showing the bodytrunk of the subject ps. In this case, the three-dimensional image G2may be a raycast image. The display control unit 166 may display thegenerated three-dimensional image G2 on the display 130 (3D display)(S13). The three-dimensional image G2 may be an image other than theraycast image.

The virtual probe processing unit 163 generates the 3D virtual probepr1. In the initial state, for example, the 3D virtual probe pr1 may beplaced so that a virtual light ray is projected onto a central pixel ofthe generated three-dimensional image G2 and the 3D virtual probe pr1transmits virtual ultrasound waves to a point which is first touched bythe body surface in a normal direction of the body surface pdf.Information in this initial state may be stored in the memory 150. Thedisplay control unit 166 places (disposes) the 3D virtual probe pr1 onthe visualized 3D display (that is, together with the three-dimensionalimage G2) (S14).

The image generation unit 162 derives (for example, calculates) asurface based on the coordinates of the virtual probe pr (3D virtualprobe pr1). The image generation unit 162 visualizes the derived surfacein accordance with an MPR method (S15). The derived surface is the MPRsurface SF. That is, the image generation unit 162 may generate 2Ddisplay of the MPR image G1 showing the MPR surface SF in the body trunkof the subject ps. The display control unit 166 may display thegenerated MPR image G1 on the display 130 (2D display) (S15). Atwo-dimensional image other than the MPR image G1 may be used.

The virtual probe processing unit 163 generates the 2D virtual probepr2. The 2D virtual probe pr2 uses the coordinates of the 3D virtualprobe pr1. The display control unit 166 shows (disposes) and displaysthe 2D virtual probe pr2 on a two-dimensional display (that is, togetherwith the MPR image G1) (S16).

The operation information acquisition unit 164 acquires variousoperations on the MPR image G1 through the UI 120. That is, theoperation information acquisition unit 164 manipulates the MPR image.The image generation unit 162 generates a new MPR image G1 based on theacquired various operations (based on the manipulated MPR image) (S17).The various operations may include a slice paging operation, a panningoperation, a rotation operation, and the like. The virtual probeprocessing unit 163 generates a new 3D virtual probe pr1 and a new 2Dvirtual probe pr2 based on the acquired various operations (S17).

The display control unit 166 updates the display of the 3D virtual probepr1 in the 3D display (S18). That is, the display control unit 166displays the generated new 3D virtual probe pr1 together with thegenerated new three-dimensional image G2. The display control unit 166updates the display of the 2D virtual probe pr2 in the 2D display (S18).That is, the display control unit 166 displays the generated new 2Dvirtual probe pr2 together with the generated new MPR image G1. In acase where the three-dimensional image G2 itself is not rotated, and thelike, the three-dimensional image G2 may not be regenerated, and theoriginal three-dimensional image G2 may be used.

FIG. 9 is a diagram for supplementing description of operations of themedical image processing apparatus 100 according to each operation onthe MPR image G1. In FIG. 9 , positions and directions in thethree-dimensional subject ps and the two-dimensional MPR image G1 areshown. In FIG. 9 , a position in a three-dimensional space is indicatedby (x, y, and z). In FIG. 9 , a position in a two-dimensional plane ofthe MPR image G1 is indicated by (u, v).

In FIG. 9 , a normal (normal line) of the MPR surface SF in the subjectps obliquely viewed is indicated by a normal vector N (x, y, z) (alsoreferred to simply as “N”). Directions of the 3D virtual probe pr1 andthe 2D virtual probe pr2 are indicated by a vector D (x, y, z) (alsoreferred to simply as “D”) in the three-dimensional space and areindicated by a vector Dmpr (u, v) (also referred to simply as “Dmpr”) inthe two-dimensional plane of the MPR image G1. Central coordinates ofthe surface of each of the 3D virtual probe pr1 and the 2D virtual probepr2 which come into contact with the subject ps are indicated bycoordinates P (x, y, z) (also referred to simply as “P”) in thethree-dimensional space and are indicated by coordinates Pmpr (u, v)(also referred to simply as “Pmpr”) in the two-dimensional plane of theMPR image G1.

An initial value of each data may be indicated by attaching “0” to theend of each variable. For example, an initial value of the coordinates Pis indicated by P0, an initial value of the coordinates Pmpr isindicated by Pmpr, an initial value of the vector D is indicated by D0,and an initial value of the vector Dmpr is indicated by Dmpr0.

Variables shown in FIG. 9 are used in the description of FIGS. 10, 15and 19 .

FIG. 10 is a flowchart showing an example of operations of the medicalimage processing apparatus 100 during a slice paging operation on theMPR image G1.

The processing unit 160 performs initial setting of various parameters(S21). That is, the processing unit 160 sets initial values for thecentral coordinates P0 of the surface of the virtual probe pr whichcomes into contact with the subject ps, the direction D of the virtualprobe pr, and the normal vector N of the MPR surface SF.

The UI 120 receives an operation (slice paging operation) to move theMPR image G1 (MPR surface SF) in an N direction by a distance s (S22).The operation information acquisition unit 164 acquires information ofthe slice paging operation through the UI 120.

The image generation unit 162 derives (for example, calculates) thecentral coordinates P of a new surface of the virtual probe pr whichcomes into contact with the subject ps based on the slice pagingoperation (S23). In this case, the image generation unit 162 may set anintersection point with the contour (that is, the body surface psf) ofthe body trunk of the subject ps, which passes through coordinates (alsoreferred to as coordinates P0+sN) moved by s in the N direction from thecoordinates P0 and which is positioned on a straight line parallel tothe direction D of the virtual probe pr, to be the central coordinates Pof the new surface which comes into contact with the subject ps of thevirtual probe pr.

In a case where a plurality of central coordinates P described above arederived in S23, the image generation unit 162 may select coordinatesclose to the coordinates P0+sN among the plurality of centralcoordinates P. In this case, for example, the medical image processingapparatus 100 can determine so that the central coordinates Pcontinuously move on the body surface psf. That is, the medical imageprocessing apparatus 100 can prevent the virtual probe pr fromdiscontinuously moving on the body surface psf. For example, when volumedata is obtained by a CT image using the CT device 200, it is assumedthat an arm portion is present together with the body trunk in atransmission direction of virtual ultrasound waves. In this case, themedical image processing apparatus 100 can prevent the centralcoordinates P from discontinuously moving from the body trunk to the armportion to perform adjustment so that the central coordinates Pcontinuously move in the body trunk. Accordingly, for example, even whenan arm of a patient is included in volume data, the medical imageprocessing apparatus 100 can adjust so that the central coordinates Pmove continuously in the body trunk.

The image generation unit 162 determines a new MPR surface SF based onthe central coordinates P, the direction D of the virtual probe pr, andthe normal vector N to generate a new MPR image G1 of the new MPRsurface SF (S24).

The virtual probe processing unit 163 generates a new 3D virtual probepr1 and a 2D virtual probe pr2 based on the central coordinates P, thedirection D of the virtual probe pr, and the normal vector N. Thedisplay control unit 166 displays the new 3D virtual probe pr1 on thethree-dimensional image G2 (S25). The display control unit 166 displaysthe new 2D virtual probe pr2 on the MPR image G1 (S25).

In a case where a slice paging operation on the MPR image G1 iscontinued, information of the slice paging operation is continuouslyacquired by the operation information acquisition unit 164 through theUI 120. In this case, the processing unit 160 may repeat the processesof S22 to S25.

In this manner, in a case where the MPR image G1 is moved in a depthdirection (slice paging is performed) through operations during theslice paging operation on the MPR image G1, the medical image processingapparatus 100 moves the virtual probe pr (3D virtual probe pr1) to theposition of the body surface in the moved MPR image G1. Accordingly, themedical image processing apparatus 100 can move the virtual probe pralong the body surface without separating the virtual probe pr from thebody surface psf even after the movement of the virtual probe.Accordingly, the user can simply obtain an image of the same area as ina case where ultrasound diagnosis is performed by sliding on the bodysurface psf with the slice paging operation on the MPR image G1 as astarting point.

FIG. 11 is a diagram illustrating slice paging according to a slicepaging operation in a comparative example. In the comparative example,in a case where the same target as a target in the description of thepresent embodiment is described, description will be given by attaching“x” to the end of a reference character in the description of thepresent embodiment. This is the same as in description of thecomparative example related to other operations. Reference charactersappearing in the comparative example may not be shown in the drawing.

In FIG. 11 , an MPR surface SFx has a predetermined angle with respectto a body surface psfx of a subject psx without being perpendicularthereto. This shows irregularities of the surface of a body. When slicepaging is performed on an MPR surface SF11 x, the MPR surface SFx ischanged to MPR surfaces SF12 x, SF13 x, . . . parallel to the MPRsurface SF11 x. Accordingly, a MPR image G1 x is changed so as to bemoved in parallel in a depth direction or a front direction in the MPRimage G1 x.

The depth direction and the front direction in the MPR image G1 x aredirections along an arrow αx. The arrow ax is not parallel to a verticaldirection of the body surface psfx in FIG. 11 . Accordingly, in a casewhere a 3D virtual probe pr1 x is not designed to move along the bodysurface psfx, the 3D virtual probe moves along the arrow αx to advanceto the inside or outside of the subject psx and separates from the bodysurface psfx. For this reason, there is a possibility that the MPR imageG1 x of the user's unintended area in the subject psx may be displayed.Accordingly, it is not appropriate to examine a place to be touched byan ultrasound probe due to the virtual probe pr separating from the bodysurface.

FIG. 12 is a diagram illustrating slice paging according to a slicepaging operation in the present embodiment. In FIG. 12 , the MPR surfaceSF has a predetermined angle with respect to the body surface psf of thesubject ps without being perpendicular thereto. When slice paging isperformed on an MPR surface SF11, the MPR surface SF is changed to MPRsurfaces SF12, SF13, . . . parallel to the MPR surface SF11.

In a case where slice paging is performed on the MPR image G1, that is,in a case where the MPR image is changed in a depth direction or a frontdirection in the MPR image G1, the virtual probe processing unit 163moves the 3D virtual probe pr1 and the 2D virtual probe pr2 to anintersection point (that is, the position of the body surface in the MPRimage G1) between the MPR image G1 and the body surface psf which areobtained by performing slice paging. Accordingly, in FIG. 12 , the 3Dvirtual probe pr1 moves along the body surface psf without beingseparating from the body surface psf. That is, the 3D virtual probe pr1moves on the body surface psf along the arrow α, and thus the MPR imageG1 of the user's intended area in the subject ps is displayed.

The virtual probe processing unit 163 may fix an angle at which the 3Dvirtual probe pr1 touches the body surface of the subject ps. That is,an angle between the body surface of the subject ps and a transmissiondirection of virtual ultrasound waves may be fixed. In this manner, themedical image processing apparatus 100 can show a tracing motion whilechanging the direction of the ultrasound probe in accordance with acurved surface (roundness) of the body surface psf, and thus the medicalimage processing apparatus approaches the movement of the ultrasoundprobe depending on an operator. The direction of touching of the 3Dvirtual probe pr1 may be fixed. This direction may indicate atransmission direction of virtual ultrasound waves. In this manner, themedical image processing apparatus 100 can be maintained in parallel tothe MPR surface SF regardless of the curved surface (roundness) of thebody surface psf, and thus it is possible to reduce the user's oversightof a disease or the like on the MPR image G1. The direction of the 3Dvirtual probe pr1 may be maintained so that the MPR surface SFnecessarily includes a target (an observation target such as a disease).An angle at which the subject ps is touched by the 3D virtual probe pr1may correspond to a direction in which virtual ultrasound waves aretransmitted and move.

FIG. 13 is a diagram showing an example of transition of the MPR imageG1A (G11A, G12A, G13A) and the 2D virtual probe pr2 according to a slicepaging operation on the MPR image G1A. FIG. 14 is a diagram showing anexample of transition of the three-dimensional image G2A (G21A, G22A,G23A) and the 3D virtual probe pr1 according to a slice paging operationon the MPR image G1A.

In FIG. 13 , the MPR image G1A is changed in order of the MPR imagesG11A, G12A, and G13A in accordance with a slice paging operation. InFIG. 14 , the three-dimensional image G2A is changed in order of thethree-dimensional images G21A, G22A, and G23A in accordance with a slicepaging operation. The MPR image G11A and the three-dimensional imageG21A indicate the subjects ps at the same timing, the MPR image G12A andthe three-dimensional image G22A indicate the subjects ps at the sametiming, and the MPR image G13A and the three-dimensional image G23Aindicate the subjects ps at the same timing.

In FIG. 13 , The 2D virtual probe pr2 does not seem to be enough contactwith the body surface psf of the subject ps, but there is a fat layer ofthe subject ps in a black portion in the vicinity of a white portionhaving high luminance. That is, the 2D virtual probe pr2 is in contactwith the body surface psf of the subject ps in all of the MPR imagesG11A, G12A, and G13A.

Referring to FIGS. 13 and 14 , even when a slice paging operation isperformed on the MPR image G1A, the medical image processing apparatus100 prevents the virtual probe pr (the 3D virtual probe pr1 and the 2Dvirtual probe pr2) indicating the position and direction of the MPRimage G1A of the MPR surface SF from separating from the body surfacepsf, so that it can be understood that the virtual probe moves on thebody surface psf. Accordingly, the medical image processing apparatus100 can operate an ultrasound image side obtained in the actualultrasound inspection to update the position and direction of the 3Dvirtual probe pr1 indicating a position and a direction in athree-dimensional space while following the operation. Accordingly, theuser can observe and confirm areas in the entire subject ps from abovewhile performing a fine operation in the MPR image G1A. An example inwhich the MPR image G1 is moved in parallel in the depth direction hasbeen described as a slice paging operation, but the virtual probeprocessing unit 163 may move the MPR image in parallel whileaccompanying panning so that the 2D virtual probe pr2 does not move onthe image. The virtual probe processing unit 163 may fix an angle atwhich the body surface of the subject ps is touched by the 3D virtualprobe pr1 and move the MPR image G1 in the depth direction as a slicepaging operation.

FIG. 15 is a flowchart showing an example of operations of the medicalimage processing apparatus 100 during a panning operation for the MPRimage G1.

The processing unit 160 performs initial setting of various parameters(S31). That is, the processing unit 160 sets initial values for thecentral coordinates P0 of the surface of the virtual probe pr whichcomes into contact with the subject ps, the direction D of the virtualprobe pr, and the normal vector N of the MPR surface SF.

The UI 120 receives an operation (panning operation) to move the MPRimage G1 by a vector S in a plane of the MPR surface SF (S32). Theoperation information acquisition unit 164 acquires information of thepanning operation through the UI 120.

The image generation unit 162 derives (for example, calculates) thecentral coordinates P of a new surface of the virtual probe pr whichcomes into contact with the subject ps based on the panning operation(S33). In this case, the image generation unit 162 may set anintersection point with the contour (that is, the body surface psf) ofthe body trunk of the subject ps, which passes through coordinates (alsoreferred to as coordinates P0-S) moved by S in a direction opposite tothe moving direction of S in S32 from the coordinates P0 and which ispositioned on a straight line parallel to the direction D of the virtualprobe pr, to be the central coordinates P of the new surface which comesinto contact with the subject ps of the virtual probe pr.

In this manner, a display range of the MPR image G1 is moved in a planeof the MPR surface SF by operating the MPR image G1, but the position ofthe central coordinates P is moved at the same distance as the movementdistance of the MPR image G1 in a direction opposite to the movingdirection of the MPR image G1. For this reason, it looks as if theposition of the 2D virtual probe pr2 is not enough moved on the MPRimage G1.

In a case where a plurality of central coordinates P described above arederived in S33, the image generation unit 162 may select coordinatesclose to the coordinates P0-S among the plurality of central coordinatesP. In this case, for example, the medical image processing apparatus 100can make determination so that the central coordinates P continuouslymove on the body surface psf. That is, the medical image processingapparatus 100 can prevent the virtual probe pr from discontinuouslymoving on the body surface psf. For example, when volume data isobtained by a CT image using the CT device 200, it is assumed that anarm portion is present together with the body trunk in a transmissiondirection of virtual ultrasound waves. In this case, the medical imageprocessing apparatus 100 can prevent the central coordinates P fromdiscontinuously moving from the body trunk to the arm portion to performadjustment so that the central coordinates P continuously move in thebody trunk.

In a case where the central coordinates P are not present in S33, theimage generation unit 162 may set a point closest to the coordinatesP0-S among points on the contour (that is, the body surface psf) of thebody trunk to be the central coordinates P of the new surface whichcomes into contact with the subject ps of the virtual probe pr. In thiscase, the medical image processing apparatus 100 can make the 2D virtualprobe pr2 follow the body surface psf by moving the position of the 2Dvirtual probe pr2 so that the 2D virtual probe pr2 does not separatefrom the body surface psf even when an operation amount of the panningoperation is large and the position of the coordinates P0-S is notpresent on the moved MPR image G1. Although an example in which thedirection Dmpr of the 2D virtual probe pr2 is fixed on the MPR image G1has been described, the virtual probe processing unit 163 may fix anangle at which the body surface of the subject ps is touched by the 3Dvirtual probe pr1 and rotate the direction Dmpr of the 2D virtual probepr2.

The image generation unit 162 determines a new MPR surface SF based onthe central coordinates P, the direction D of the virtual probe pr, andthe normal vector N to generate a new MPR image G1 of the new MPRsurface SF (S34).

The virtual probe processing unit 163 generates a new 3D virtual probepr1 and 2D virtual probe pr2 based on the central coordinates P, thedirection D of the virtual probe pr, and the normal vector N. Thedisplay control unit 166 displays the new 3D virtual probe pr1 on thethree-dimensional image G2 (S35). The display control unit 166 displaysthe new 2D virtual probe pr2 on the MPR image G1 (S35).

In a case where a panning operation for the MPR image G1 is continued,information of the panning operation is continuously acquired by theoperation information acquisition unit 164 through the UI 120. In thiscase, the processing unit 160 may repeat the processes of S32 to S35.

In this manner, the medical image processing apparatus 100 moves the 3Dvirtual probe pr1 and the 2D virtual probe pr2 so as to slide the bodysurface of the subject ps according to an operation during the panningoperation for the MPR image G1. In this case, the medical imageprocessing apparatus 100 can make the 3D virtual probe pr1 and the 2Dvirtual probe pr2 follow the body surface by finely adjusting theposition of the central coordinates P. Accordingly, the medical imageprocessing apparatus 100 can move the virtual probe pr along the bodysurface without separating the virtual probe pr from the body surfacepsf even after the movement of the virtual probe. Accordingly, the usercan easily obtain an image of the same area as in a case whereultrasound diagnosis is performed by sliding the virtual probe on thebody surface psf, with the panning operation for the MPR image G1 as astarting point.

FIG. 16 is a diagram showing transition of an MPR image G1Bx (G11Bx,G12Bx, G13Bx) and a 2D virtual probe pr2 x according to a panningoperation in a comparative example.

In FIG. 16 , when the MPR image G1Bx is moved in any direction within anMPR surface SFx according to a panning operation, the position of eachpoint on the MPR image G11Bx displayed on the display 130 is moved. InFIG. 16 , the MPR image G1Bx is changed in order of the MPR imagesG11Bx, G12Bx, G13Bx, . . . . In this case, the 2D virtual probe pr2 x ismoved according to movement in a plane the MPR surface SFx. That is, adisplay position of the 2D virtual probe pr2 x on the display 130changes similarly in accordance with a panning operation. For thisreason, it is not possible to move the 2D virtual probe pr2 x accordingto a panning operation.

As a comparative example, in a case where the MPR image G1Bx is moved inany direction within the MPR surface SFx according to a panningoperation, the 2D virtual probe pr2 x may not follow and move at all andthe display position of the 2D virtual probe pr2 x on the display 130may not move at all and remains stationary. In this case, the 2D virtualprobe pr2 x separates from the body surface psf, and thus it is notappropriate to examine a place to be touched by an ultrasound probe.

FIG. 17 is a diagram showing an example of transition of an MPR imageG1B (G11B, G12B, G13B) and the 2D virtual probe pr2 according to apanning operation for the MPR image G1B in the present embodiment. FIG.18 is a diagram showing an example of transition of a three-dimensionalimage G2B (G21B, G22B, G23B) and the 3D virtual probe pr1 according to apanning operation for the MPR image G1B in the present embodiment.

In FIG. 17 , when the MPR image G11B is moved in any direction withinthe MPR surface SF according to a panning operation, the position ofeach point of the MPR image G11B displayed on the display 130 is moved.In FIG. 17 , the MPR image G1B is moved in order of the MPR images G11B,G12B, G13B, . . . . In this case, the 2D virtual probe pr2 is moved atthe same movement distance in a direction opposite to a direction ofparallel movement of the MPR image G1 within the MPR surface SF. The 2Dvirtual probe pr2 is moved while maintaining a state where the 2Dvirtual probe pr2 is in contact with a point on the body surface psf.

In this manner, in FIG. 17 , in a case where a panning operation isperformed on the MPR image G1B, the display control unit 166 moves anddisplays the virtual probe pr so as to slide on the body surface psf. Inthis case, the center of the 2D virtual probe pr2 in a lateral directionis fixed by being aligned with the center of the MPR image G1B in alateral direction, and the center of the 2D virtual probe pr2 in avertical direction is moved along the body surface.

Accordingly, the medical image processing apparatus 100 can maintain acontact state between the 2D virtual probe pr2 and the body surface psfwhile suppressing a change in the display position of the 2D virtualprobe pr2 with respect to the display 130 if possible. Therefore, themedical image processing apparatus 100 can use the 2D virtual probe pr2after operation to examine a place to be touched by an ultrasound probe.

In FIG. 18 , when the MPR image G11B is moved in any direction withinthe MPR surface SF according to a panning operation, thethree-dimensional image G21B is not moved, and a display position of the3D virtual probe pr1 changes. That is, it shows that the position oftransmission of virtual ultrasound waves transmitted from the 3D virtualprobe pr1 changes in accordance with a panning operation. In FIG. 18 ,the three-dimensional image G2B transitions in order of thethree-dimensional images G21B, G22B, G23B, . . . which correspond to theMPR images G11B, G12B, G13B, . . . , respectively. Accordingly, the usercan easily ascertain the position and direction of the MPR image G1Bwith respect to the three-dimensional image G2B by confirming theposition and direction of the 3D virtual probe pr1 together with thethree-dimensional image G2B.

In a case where the direction Dmpr of the virtual probe pr faces in a −vdirection (see FIG. 9 ) and the MPR image G1B is moved in parallel in av direction along the MPR surface SF in accordance with a panningoperation, the virtual probe pr may be moved in a +v direction. In acase where the direction Dmpr of the virtual probe pr faces in the −vdirection and the MPR image G1B is moved in parallel in a u directionperpendicular to the v direction along the MPR surface SF in accordancewith a panning operation, the virtual probe pr is not moved in the udirection and may be moved in the v direction along the body surfacepsf.

FIG. 19 is a flowchart showing an example of operation of the medicalimage processing apparatus 100 during a rotation operation for the MPRimage G1.

The processing unit 160 performs initial setting of various parameters(S41). That is, the processing unit 160 sets initial values for thecentral coordinates P0 of the surface of the virtual probe pr whichcomes into contact with the subject ps, the direction D of the virtualprobe pr, the normal vector N of the MPR surface SF, the centralcoordinates Pmpr in a two-dimensional plane (MPR surface SF) of thesurface of the virtual probe pr which comes into contact with thesubject ps, and the direction Dmpr0 in a two-dimensional plane (MPRsurface SF) of the virtual probe pr.

The UI 120 receives an operation (rotation operation) to rotate the MPRimage G1 by an angle Ψ within the MPR surface SF (S42). The operationinformation acquisition unit 164 acquires information of the rotationoperation through the UI 120.

The image generation unit 162 derives (for example, calculates) thedirection Dmpr of the virtual probe pr in a two-dimensional plane (MPRsurface SF) based on the rotation operation (S43). In this case, theimage generation unit 162 may calculate the direction Dmpr of the 2Dvirtual probe pr2 in a case where the MPR image is rotated by the angleΨ in a two-dimensional plane from the direction Dmpr0 of the 2D virtualprobe pr2 before the rotation operation.

The image generation unit 162 derives (for example, calculates) thedirection D of the virtual probe pr in a three-dimensional space basedon the rotation operation (S44). In this case, the image generation unit162 may calculate the direction D of the 3D virtual probe pr1 in athree-dimensional space in a case where the MPR image is rotated by theangle Ψ in a three-dimensional space from the direction D0 of the 3Dvirtual probe pr1 before the rotation operation by using the normalvector N of the MPR surface SF as an axis.

The image generation unit 162 generates a new MPR image G1 rotatedwithin the MPR surface SF based on the central coordinates P0, thedirection D of the virtual probe pr, and the normal vector N (S45).

The virtual probe processing unit 163 generates a new 3D virtual probepr1 based on the central coordinates P0, the direction D of the virtualprobe pr, and the normal vector N. The virtual probe processing unit 163generates a new 2D virtual probe pr2 based on the central coordinatesP0, the direction Dmpr of the virtual probe pr, and the normal vector N.The display control unit 166 displays the new 3D virtual probe pr1 onthe three-dimensional image G2 (S46). The display control unit 166displays the new 2D virtual probe pr2 on the MPR image G1 (S46).

In a case where a rotation operation for the MPR image G1 is continued,information of the rotation operation is continuously acquired by theoperation information acquisition unit 164 through the UI 120. In thiscase, the processing unit 160 may repeat the processes of S42 to S46.

In this manner, the medical image processing apparatus 100 candetermine, for example, a contact point between the virtual probe pr andthe body surface psf in accordance with an operation during the rotationoperation for the MPR image G1 and rotate the MPR image G1 around thecontact point on the body surface. That is, the position of the centralcoordinates P may not change. It is possible to update the display ofthe virtual probe pr without making the virtual probe pr follow the bodysurface during the rotation operation.

FIG. 20 is a diagram showing transition of an MPR image G1Cx (G11Cx,G12Cx, G13Cx) and a 2D virtual probe pr2 x according to a rotationoperation in a comparative example.

In FIG. 20 , when the MPR image G1Cx is rotated within the MPR surfaceSFx according to a rotation operation, the position of each point of theMPR image G1Cx displayed on the display 130 is rotated centering on areference point on the MPR image G1Cx. In FIG. 16 , the transition isperformed in order of the MPR images G11Cx, G12Cx, G13Cx, . . . . Inthis case, the 3D virtual probe prix on a three-dimensional image G2Cxis rotated in association with the rotation of the body surface psfx ina plane of the MPR surface SFx. That is, the direction of the 2D virtualprobe pr2 x on the display 130 changes in accordance with the rotationoperation, similar to the rotation of the MPR image G1Cx. For thisreason, it is not possible to rotate the 2D virtual probe pr2 x inaccordance with a rotation operation for the MPR image G1.

FIG. 21 is a diagram showing an example of transition of an MPR imageG1C (G11C, G12C, G13C) and the 2D virtual probe pr2 according to arotation operation for the MPR image G1C in the present embodiment. FIG.22 is a diagram showing an example of transition of a three-dimensionalimage G2C (G21C, G22C, G23C) and the 3D virtual probe pr1 according to arotation operation for the MPR image G1C in the present embodiment.

In FIG. 21 , when the MPR image G11C is rotated in any direction withinthe MPR surface SF according to a rotation operation, the position ofeach point of the MPR image G11B displayed on the display 130 is rotatedcentering on the position of the 2D virtual probe pr2 (for example, theposition of transmission (the central coordinates P of the lower surfaceds) of ultrasound waves of the 2D virtual probe pr2). Here, the centerof rotation is not limited to one point of the central coordinates P ofthe lower surface ds, and may be a range in the vicinity of the centralcoordinates P of the lower surface ds or may have a certain degree ofwidth from the central coordinates P of the lower surface ds. In FIG. 21, the MPR image G1C transitions in order of the MPR images G11C, G12C,G13C, . . . . On the other hand, in FIG. 21 , the 2D virtual probe pr2is not rotated regardless of the rotation of the MPR image G1 within theMPR surface SF. That is, in FIG. 21 , a transmission direction ofvirtual ultrasound waves is a lower direction which is fixed in the 2Dvirtual probe pr2. Therefore, the 2D virtual probe pr2 is rotatedcounterclockwise (leftward) with respect to the body surface psf of thesubject ps. However, in FIG. 21 , it looks as if the 2D virtual probepr2 is not rotated and the subject ps is relatively rotated clockwise(rightward) with respect to the 2D virtual probe pr2.

The 2D virtual probe pr2 is rotated while maintaining a state where the2D virtual probe pr2 is in contact with a point on the body surface psf.Accordingly, the medical image processing apparatus 100 can confirm thesubject ps which is an observation target from various angles along theMPR surface SF and can maintain a contact state between the body surfacepsf and the 2D virtual probe pr2.

Therefore, the medical image processing apparatus 100 can improvedisplay accuracy of the position and direction of the 2D virtual probepr2 after operation and can prevent relative positions and directionsbetween the three-dimensional image G2 and the MPR image G1 after arotation operation from being deviated.

In FIG. 22 , when the MPR image G11C is rotated in any direction withinthe MPR surface SF according to a rotation operation, thethree-dimensional image G21C is not rotated and the display direction ofthe 3D virtual probe pr1 changes. That is, it is indicated that atransmission direction of virtual ultrasound waves transmitted from the3D virtual probe pr1 changes in accordance with a rotation operation. InFIG. 21 , the three-dimensional image G2C transitions in order of thethree-dimensional images G21C, G22C, G23C, . . . which correspond to theMPR images G11C, G12C, G13C, . . . , respectively. Accordingly, the usercan easily ascertain the position and direction of the MPR image G1Cwith respect to the three-dimensional image G2C by confirming theposition and direction of the 3D virtual probe pr1 together with thethree-dimensional image G2C.

In this manner, according to the medical image processing apparatus 100of the present embodiment, the user can update the display of athree-dimensional image shown by the MPR image G1 or the virtual probepr indicating a positional relationship between the MPR image G1 and thethree-dimensional image G2 by operating the position and direction ofthe MPR image G1 through the UI 120. Accordingly, the medical imageprocessing apparatus 100 can indicate which position and direction inthe three-dimensional image the position and direction of the MPR imageG1 indicate based on the position and direction of the virtual probe prin accordance with a received operation (for example, a slice pagingoperation, a panning operation, or a rotation operation). Accordingly,the user can easily ascertain the positional relationship by conformingthe direction and position of the virtual probe pr.

Accordingly, for example, the user observes the inside of the subject psin the MPR image G1 displayed on the display 130 and can observe an arearequired to be checked in detail from above in the three-dimensionalimage G2 displayed on the display 130 in a case where the area ispresent.

The medical image processing apparatus 100 can be used for imageprocessing for performing virtual ultrasound diagnosis. The virtualultrasound diagnosis may include virtual transesophageal ultrasounddiagnosis.

Up to here, although various embodiments have been described withreference to the accompanying drawings, it is needless to say that thepresent disclosure is not limited to such examples. It would be apparentfor those skilled in the art that various modification examples orcorrected examples are conceivable within the scope recited in theclaims, and it would be understood that these fall within the technicalscope of the invention.

In the first embodiment, the region extraction unit 161 may extract abody trunk except for an arm portion of the subject ps. That is, theregion extraction unit 161 may execute an algorithm for extracting thebody trunk except for the arm portion. Thereby, for example, it ispossible to prevent the virtual probe pr from being discontinuouslymoved from a trunk portion included in the body trunk to the arm portionin accordance with the user's operation for the MPR image G1.

In the first embodiment, an example in which the region extraction unit161 collectively extracts the entire contour (that is, the body surfacepsf) of the body trunk of the subject ps has been described, but thedisclosure is not limited thereto. In a case where information of anoperation for the MPR image G1 is acquired by the operation informationacquisition unit 164, the region extraction unit 161 may extract thecontour of the body trunk by limiting a range to the contour of the bodytrunk in the vicinity of the virtual probe pr. The region extractionunit 161 may sequentially extract the contours of the body trunk in thevicinity of the moved virtual probe pr in accordance with an operationfor the MPR image G1. Thereby, for example, the extraction of a contourcan be omitted with respect to a portion having a contour which is notnecessary for the derivation of a contact point between the virtualprobe pr and the human body among the contours of the body trunk of thesubject ps. Also in this case, the medical image processing apparatus100 can reduce the amount of arithmetic operation performed by theprocessing unit 160 while specifying the position and direction of the3D virtual probe pr1 on the MPR image G1 or the three-dimensional imageG2.

In the first embodiment, an example in which the region extraction unit161 extracts the contour of the body trunk in accordance with regiongrowing has been described, but the contour of the body trunk may beextracted using other methods. For example, the region extraction unit161 may derive (for example, calculate) coordinates where reflectedlight used in deriving a raycast image is generated or coordinates wherereflected light is accumulated by a threshold value th1 or greater asthe contour of the body trunk. Thereby, the medical image processingapparatus 100 can share a portion of arithmetic operation for derivingthe contour of the body trunk and arithmetic operation for generating araycast image and can reduce the amount of arithmetic operation of theprocessing unit 160.

In the first embodiment, a shape shown in FIG. 4A is exemplified as theshape of the 3D virtual probe pr1 on the three-dimensional image G2, butthe disclosure is not limited thereto. For example, the 3D virtual probepr1 may be indicated by arrows representing the position of transmissionof virtual ultrasound waves (transmission starting position) and atransmission direction thereof. The 3D virtual probe pr1 may beindicated by two points representing a transmission starting position ofvirtual ultrasound waves and a passage position of virtual ultrasoundwaves. For example, the 3D virtual probe pr1 may be indicated by twopoints representing one point of a transmission starting position ofvirtual ultrasound waves and one point within a target. In these cases,it is possible to sufficiently confirm the position and direction oftransmission of virtual ultrasound waves and to simplify the 3D virtualprobe pr1.

In the first embodiment, a shape shown in FIG. 4B is exemplified as theshape of the 2D virtual probe pr2 on the MPR image G1, but thedisclosure is not limited thereto. For example, the 2D virtual probe pr2may be indicated by arrows representing the position of transmission ofvirtual ultrasound waves (transmission starting position) in the MPRimage G1 and a transmission direction thereof. In a case where atransmission direction (for example, a vertical direction on the image)of ultrasound waves on the MPR image G1 is fixedly determined inadvance, the 2D virtual probe pr2 may be indicated by one pointrepresenting the position of transmission of virtual ultrasound waves.Accordingly, it is possible to simplify the 2D virtual probe pr2.

The position of transmission of virtual ultrasound waves and atransmission direction thereof are shown indirectly by using coordinatesof a center point of the MPR image G1, so that the 2D virtual probe pr2may be clearly shown on the MPR image G1. The indirect visualization mayinclude showing a line of a puncture on the display 130, showing afan-shaped frame indicating a virtual ultrasound image on the display130, and the like.

In this manner, the 2D virtual probe pr2 may be or may not be displayedon the display 130. The display control unit 166 may determine whetherto display the 2D virtual probe pr2 or not or may switch between displayand non-display. Thereby, the medical image processing apparatus 100 candetermine whether to display the 2D virtual probe pr2 or not inaccordance with the user's intention.

In the first embodiment, in a case where the display of thethree-dimensional image G2 and the 3D virtual probe pr1 is updated inaccordance with an operation for the MPR image G1, the virtual probeprocessing unit 163 may generate the 3D virtual probe pr1 so that anangle between the direction D of the 3D virtual probe pr1 on thethree-dimensional image G2 and the direction of a normal line of thebody surface psf of the subject ps is maintained. The direction D of the3D virtual probe pr1 may be parallel to the MPR surface SF. Thedirection D of the 3D virtual probe pr1 and the direction of the normalline of the body surface psf of the subject ps may be or may not beparallel to each other.

In this case, the virtual probe processing unit 163 may rotate thedirection of the 3D virtual probe pr1 (that is, may make the directionvariable) in a case where the body trunk of the subject ps is regardedas a cylinder and the 3D virtual probe pr1 is moved in a circumferentialdirection of the body trunk. The virtual probe processing unit 163 mayfix the direction of the 3D virtual probe pr1 in a case where the bodytrunk of the subject ps is regarded as a cylinder and the 3D virtualprobe pr1 is moved in an axial direction of the body trunk. In contrast,the direction of the 3D virtual probe pr1 may be fixed in a case wherethe 3D virtual probe pr1 is moved in the circumferential direction ofthe body trunk, and the direction of the 3D virtual probe pr1 may bevariable in a case where the 3D virtual probe pr1 is moved in the axialdirection of the body trunk.

The virtual probe processing unit 163 may rotate the direction of anormal line of the body surface of the subject ps (that is, may make thedirection variable) in a case where the body trunk is regarded as acylinder and the 3D virtual probe pr1 is moved in the circumferentialdirection of the body trunk. The virtual probe processing unit 163 mayfix the direction of the normal line of the body surface of the subjectps in a case where the body trunk is regarded as a cylinder and the 3Dvirtual probe pr1 is moved in the axial direction of the body trunk. Incontrast, the direction of the normal line of the body surface of thesubject ps may be fixed in a case where the 3D virtual probe pr1 ismoved in the circumferential direction of the body trunk, and thedirection of the normal line of the body surface of the subject ps maybe variable in a case where the 3D virtual probe pr1 is moved in theaxial direction of the body trunk.

Thereby, the virtual probe processing unit 163 may determine whether tomaintain the above-described angle or not. The user can easily recognizethe direction of the MPR image G1 with respect to the three-dimensionalsubject ps without depending on the operation for the MPR image G1 bymaintaining the above-described angle. The user can observe the subjectps while variously changing the direction of the MPR image G1 withrespect to the three-dimensional subject ps in accordance with anoperation for the MPR image G1 by not maintaining the above-describedangle.

The first embodiment may be applied to a lumen of the subject ps insteadof the body surface of the subject ps. That is, when the MPR image G1 ismoved or rotated in accordance with an operation for the MPR image G1,the 3D virtual probe pr1 and the 2D virtual probe pr2 may be moved orrotated along the lumen while being in contact with the lumen. The bodysurface of the subject ps is not limited to the body trunk and may be ahead portion or a body surface of a limb. The body surface may be thesurface of an organ instead of the body surface of the subject ps. Thebody trunk of the subject ps may be a portion ranging from an abdomen toa thoracic neck.

In the first embodiment, the image generation unit 162 may visualizeusing any method such as a raycast method, a MW method limited to a bodytrunk region, RaySUM limited to a body trunk region, or a surfacerendering method including the surface of a body trunk. There may be nolimitation to the above-mentioned body trunk region. While the 3Dvirtual probe pr1 is moved along the body surface, the image generationunit 162 may visualize only an affected area or an organ in the vicinityof the affected area.

In the first embodiment, the image generation unit 162 may visualizeusing any method such as a so-called thick MPR image, a thick MPR imagefor performing SUM within a thickness range, or a thick MIP image forperforming SUM within a thickness range as an MPR image. The imagegeneration unit 162 may visualize using a so-called pseudo ultrasoundimage subjected to processing such as simulating a reflected wave fromvolume data or adding distortion of an edge portion as an MPR image.

In the first embodiment, volume data as an obtained CT image istransmitted from the CT scanner 200 to the medical image processingapparatus 100. Alternatively, volume data may be transmitted to a serveror the like on a network and stored in the server or the like so as tobe temporarily accumulated. In this case, the port 110 of the medicalimage processing apparatus 100 may acquire volume data from the serveror the like when necessary through a wired circuit or a wireless circuitor may acquire volume data through any storage medium (not shown).

In the first embodiment, volume data as an obtained CT image istransmitted from the CT scanner 200 to the medical image processingapparatus 100 through the port 110. It is assumed that this alsoincludes a case where the CT scanner 200 and the medical imageprocessing apparatus 100 are substantially combined as one product. Inaddition, this also includes a case where the medical image processingapparatus 100 is treated as a console of the CT scanner 200.

In the first embodiment, an image is obtained by the CT scanner 200 togenerate volume data including information regarding the inside of aninternal organism. However, an image may be obtained by any of otherdevices to generate volume data. Other devices include a magneticresonance imaging (MM) apparatus, a positron emission tomography (PET)device, a blood vessel angiographic device (angiography device), orother modality devices. In addition, the PET device may be used incombination with other modality devices.

In the first embodiment, a human body is described as a subject, but ananimal body may also be used.

In the present disclosure, a program for realizing functions of themedical image processing apparatus of the first embodiment is suppliedto the medical image processing apparatus through a network or variousstorage mediums, and the present disclosure is also applicable to aprogram read out and executed by a computer within the medical imageprocessing apparatus.

As described above, in the medical image processing apparatus 100 of theabove-described embodiment, an acquisition unit (for example, the port110) acquires volume data of the subject ps. The processing unit 160displays the three-dimensional image G2 by rendering volume data, on adisplay unit (for example, the display 130). The processing unit 160displays a first object (for example, the 3D virtual probe pr1) showing(i) a point (for example, the central coordinates P) on the body surfaceof the subject ps (for example, the body surface psf) and (ii) thedirection with respect to volume data (for example, the direction D ofthe virtual probe pr) on the three-dimensional image G2, on the displayunit. The processing unit 160 displays a two-dimensional image (forexample, the MPR image G1) of a surface on the display unit. The surface(for example, the MPR surface SF) includes the point on the body surfaceand is defined based on the direction, in volume data. The processingunit 160 acquires information of a first operation to change the displayof a two-dimensional image. The processing unit 160 moves a point on thebody surface along the body surface of the subject ps to update thedisplay of the first object and the two-dimensional image based on theabove-described first operation.

According to the present disclosure, the user observes the subject psusing a two-dimensional image, for example, at the time of diagnosis andeasily understands an accurate position of disease or the like in thesubject ps. The user can observe the entire subject ps from above byusing the three-dimensional image G2. In this case, the user canascertain to which position and which direction the two-dimensionalimage corresponds in the three-dimensional image G2.

The medical image processing apparatus 100 can maintain a state wherethe subject ps is easily observed by making the position of the 3Dvirtual probe pr1 on the three-dimensional image G2 follow the bodysurface of the subject ps. Accordingly, the user convenience isimproved.

In a movement operation on the two-dimensional image in a conventionalway, a two-dimensional plane is considered, and the surface of thethree-dimensional image is not usually considered. On the other hand,the medical image processing apparatus 100 can recognize a surface (thebody surface psf) in the subject ps by making the 3D virtual probe pr1follow the surface on the three-dimensional image G2 in accordance withthe operation in the two-dimensional image. The user performs theoperation on the two-dimensional image instead of the three-dimensionalimage G2, and thus the user easily recognizes a direction related tomovement and performs a fine operation.

In a case where the user operates the two-dimensional image to updatethe display thereof, it is easy to ascertain to which position and whichdirection the updated two-dimensional image corresponds in thethree-dimensional image G2.

The above-described first operation may include slice paging of thesurface on which the two-dimensional image is shown (for example, theMPR surface SF).

Even when the slice paging operation is performed on the two-dimensionalimage through an operation unit (for example, the UI 120), the medicalimage processing apparatus 100 can ascertain to which position and whichdirection the operated two-dimensional image corresponds in thethree-dimensional image.

The medical image processing apparatus 100 can easily move the positionof the virtual probe pr1 by the slice paging operation on thetwo-dimensional image through the operation unit (for example, the UI120).

The above-described first operation may include moving a display rangeof the two-dimensional image in parallel on the surface (for example,the MPR surface SF).

Even when a panning operation is performed on the two-dimensional imagethrough the operation unit (for example, the UI 120), the medical imageprocessing apparatus 100 can ascertain to which position and whichdirection the operated two-dimensional image corresponds in thethree-dimensional image.

The medical image processing apparatus 100 can easily move the positionof the virtual probe pr1 by performing the panning operation on thetwo-dimensional image through the operation unit (for example, the UI120).

The processing unit 160 may acquire information of a second operation torotate the two-dimensional image on the surface (for example, the MPRsurface SF), in addition to the first operation. The processing unit 160may fix the point on the body surface to update a first object on thethree-dimensional image and the two-dimensional image based on thesecond operation.

Even when the rotation operation is performed on the two-dimensionalimage through the operation unit (for example, the UI 120), the medicalimage processing apparatus 100 can ascertain to which position and whichdirection the operated two-dimensional image corresponds in thethree-dimensional image. The medical image processing apparatus 100 mayfix the point on the body surface and rotate the two-dimensional image,so that the user can observe the same two-dimensional image from anydirection, which facilitates observation.

The medical image processing apparatus 100 can easily move the directionof the virtual probe pr1 by the rotation operation on thetwo-dimensional image through the operation unit (for example, the UI120).

The processing unit 160 may extract volume data of the body trunk fromvolume data of the subject ps to generate the three-dimensional imageand the two-dimensional image of the volume data of the body trunk.

The medical image processing apparatus 100 can suppress the generationof a plurality of intersection points between the surface on which thetwo-dimensional image is rendered (for example, the MPR surface SF) andthe body surface of the subject ps. Accordingly, it is possible tosuppress discontinuous movement of the first object on thethree-dimensional image and a great change in an area of the subject psshown by the two-dimensional image in accordance with an operation forthe two-dimensional image. Accordingly, the user easily observes thesubject ps while performing the operation to the two-dimensional image.

The processing unit 160 may update the display of the first object andthe two-dimensional image by maintaining an angle between the directionof the vector and the direction of the normal line of the body surfacebased on the above-described operation.

The user easily recognize the direction of the MPR image G1 with respectto the three-dimensional subject ps without depending on the operationfor the MPR image G1 by the medical image processing apparatus 100maintaining the above-described angle.

The user easily observes the subject by performing a motion close to anoperation of tracing the body surface of a patient using an ultrasoundprobe by the medical image processing apparatus 100 maintaining theabove-described angle.

The processing unit 160 may update the display of the first object andthe two-dimensional image by maintaining the direction based on theabove-described operation.

The user easily recognizes the direction of the MPR image G1 for thethree-dimensional subject ps without depending on an operation for theMPR image G1 by the medical image processing apparatus 100 maintainingthe direction.

The user easily observes the subject by performing a motion close to anoperation of tracing the body surface of a patient using an ultrasoundprobe by the medical image processing apparatus 100 maintaining thedirection.

The processing unit 160 may display a second object (for example, the 2Dvirtual probe pr2) showing (iii) a point on the body surface of thesubject ps and (iv) the direction in the two-dimensional image on adisplay unit.

The medical image processing apparatus 100 can display the second objectindicating a position and a direction similar to the first object,together with the two-dimensional image. Accordingly, the user canconfirm information of the position and direction of the two-dimensionalimage in the three-dimensional image on the two-dimensional image.Therefore, the user convenience is improved.

The direction may indicate a transmission direction of virtualultrasound waves. The above-described surface may indicate the surfacealong a passage through which virtual ultrasound waves pass.

The medical image processing apparatus 100 can set the direction and thesurface related to virtual ultrasound waves. Accordingly, the usereasily ascertains to which position and which direction thetwo-dimensional image corresponds in the three-dimensional image G2, ina case where diagnosis using the virtual ultrasound image is performed.

The present disclosure is useful for a medical image processingapparatus, a medical image processing method, and a medical imageprocessing system which are capable of ascertaining to which positionand which direction a changed two-dimensional image corresponds in athree-dimensional image in a case where a user operates thetwo-dimensional image to update display thereof

What is claimed is:
 1. A medical image processing apparatus comprising:an acquisition unit that is configured to acquire volume data of asubject; and a processing unit that is configured to: display athree-dimensional image by rendering the acquired volume data, on adisplay unit; display a first user interface (UI) object showing (i) apoint on a body surface of the subject and (ii) a direction with respectto the volume data, in the three-dimensional image on the display unit;display a two-dimensional image of a cross-section on the display unit,wherein the cross-section includes the point on the body surface and isdefined based on the direction, in the volume data; acquire informationof a first operation to change display of the two-dimensional image; andmove the point on the body surface along the body surface of the subjectbased on the first operation to update display of the first UI objectand the two-dimensional image.
 2. The medical image processing apparatusaccording to claim 1, wherein the first operation includes slice pagingof the surface on which the two-dimensional image is shown.
 3. Themedical image processing apparatus according to claim 1, wherein thefirst operation includes moving a display range of the two-dimensionalimage in parallel on the surface.
 4. The medical image processingapparatus according to claim 1, wherein the processing unit isconfigured to: acquire information of a second operation to rotatearound the point on the body surface the two-dimensional image on thesurface, in addition to the first operation; and update the display ofthe first UI object on the three-dimensional image and thetwo-dimensional image based on the second operation.
 5. The medicalimage processing apparatus according to claim 1, wherein the processingunit is configured to: extract body trunk from the volume data of thesubject, and generate the three-dimensional image and thetwo-dimensional image of the body trunk.
 6. The medical image processingapparatus according to claim 1, wherein the processing unit isconfigured to maintain an angle between the direction and a direction ofa normal line with respect to the body surface to update the display ofthe first UI object and the two-dimensional image based on the firstoperation.
 7. The medical image processing apparatus according to claim1, wherein the processing unit is configured to maintain the directionto update the display of the first UI object and the two-dimensionalimage based on the first operation.
 8. The medical image processingapparatus according to claim 1, wherein the processing unit isconfigured to display a second UI object showing (iii) the point on thebody surface of the subject and (iv) the direction, in thetwo-dimensional image on the display unit.
 9. The medical imageprocessing apparatus according to claim 1, wherein the directionindicates a transmission direction of virtual ultrasound waves, and thesurface indicates a surface along a passage through which the virtualultrasound waves pass.
 10. A medical image processing method in amedical image processing apparatus, the method comprising: acquiringvolume data of a subject; displaying a three-dimensional image byrendering the acquired volume data; displaying a first user interface(UI) object showing (i) a point on a body surface of the subject and(ii) a direction with respect to the volume data in thethree-dimensional image; displaying a two-dimensional image of across-section, wherein the cross-section includes the point on the bodysurface and is defined based on the direction, in the volume data;acquiring information of an operation to change display of thetwo-dimensional image; and moving the point on the body surface alongthe body surface of the subject based on the operation to update displayof the first UI object and the two-dimensional image.
 11. A medicalimage processing system causing a medical image processing apparatus toexecute the medical image processing operations comprising: acquiringvolume data of a subject; displaying a three-dimensional image byrendering the acquired volume data; displaying a first user interface(UI) object showing (i) a point on a body surface of the subject and(ii) a direction with respect to the volume data in thethree-dimensional image; displaying a two-dimensional image of across-section, wherein the cross-section includes the point on the bodysurface and is defined based on the direction, in the volume data;acquiring information of an operation to change display of thetwo-dimensional image; and moving the point on the body surface alongthe body surface of the subject based on the operation to update displayof the first UI object and the two-dimensional image.